This document contains the results and analysis of my second experiment for my first Qualifying Paper towards the PhD in Linguistics at Stanford University1.
The experiment analyzed herein was an online maze task study, wherein participants read a sentence such as “David is a congressperson from Virginia. He likes cycling.” For each of the 20 critical items, participants were randomly assigned to one of four conditions, the critical regions of which are enumerated and exemplified here:
Participants proceeded through the sentences by pressing a keyboard key which corresponded to the grammatical continuation of the sentence, and reading pair of grammatical and distractor items at a time. They were then asked attention check questions at the end of each sentence; attention check questions never explicitly invoked gender. Rather, they asked about the character’s home states. For a hands-on look at the experiment, you can click here to go to the same webpage particpants were directed to for the task.
##Participants We originally ran the experiment on 200 participants, recruited through the online participant recruitment platform Prolific. The mean time of the experiment was 5.39 minutes, and participants were paid $1.75 for their participation2. The only restrictions placed on participants were that they:
These requirements were implemented in order to assure that speakers came from at least somewhat similar linguistic backgrounds, as certain lexical items in the study (such as congressperson) are quite localized to the United States.
After this initial run of the experiment, we found that there was a dearth of conservative or Republican-aligned participants. As a result, we ran the experiment again, this time on 98 self-identified Republicans. This was achieved by adding a filter on Prolific so that only Republican-identified individuals could see the task. The rest of the experiment, including payment, was exactly the same, except that an additional disclaimer that participants could not use the FireFox browser experiment, after the first run revealed an incompatibility between JavaScript and FireFox. The two runs of the experiment amounted in a total of 298 participants who completed the task.
Before we can do much of anything with the data, we need to make sure it’s usable! This means filtering out all unimportant or extraneous trials, running exclusion criteria, and adding additional trial and item-level data that will be necessary later in the analysis.
For this analysis, we require the following packages:
library(ggplot2)
library(tidyverse)
Registered S3 methods overwritten by 'dbplyr':
method from
print.tbl_lazy
print.tbl_sql
── Attaching packages ──────────────────────────────────────────────────────────────── tidyverse 1.3.1 ──
✓ tibble 3.1.2 ✓ dplyr 1.0.6
✓ tidyr 1.1.3 ✓ stringr 1.4.0
✓ readr 1.4.0 ✓ forcats 0.5.1
✓ purrr 0.3.4
── Conflicts ─────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
x dplyr::filter() masks stats::filter()
x dplyr::lag() masks stats::lag()
library(lme4)
Loading required package: Matrix
Attaching package: ‘Matrix’
The following objects are masked from ‘package:tidyr’:
expand, pack, unpack
library(stringr)
library(languageR)
library(lmerTest)
Attaching package: ‘lmerTest’
The following object is masked from ‘package:lme4’:
lmer
The following object is masked from ‘package:stats’:
step
library(reshape2)
Attaching package: ‘reshape2’
The following object is masked from ‘package:tidyr’:
smiths
source("helpers.R")
In the first instance, we need to read in the data, which has been pre-merged from both runs of the experiment. We will also in this chunk filter out all of the example trials, as well as all the data points that are from non-critical trials.
all_data <- read.csv('merged_all.csv') %>%
filter(trial_id!= 'example') %>%
filter(region=='critical')
Now, we want to exclude any participants who failed to answer at least 85% of the attention check questions correctly. We do this by creating a list of all participants who scored less than 85% on these checks, and then cross-referencing this list with all data points, removing any data points whose participants were in the exclusion list.
exclusion <- all_data %>% group_by(workerid) %>%
summarise(accuracy = mean(response_correct)) %>%
mutate(exclude = ifelse(accuracy < 0.85,'Yes','No')) %>%
filter(exclude == 'Yes')
all_data <- all_data[!(all_data$workerid %in% exclusion$workerid),] %>%
filter(rt !='null')
We also want to filter out all trials in which the reading time for the critical item was more than 2.5 standard deviations from the mean reading time on that lexical item across all participants.
all_data <- all_data %>% group_by(trial_id) %>% mutate(id_mean = mean(log(rt))) %>% mutate(exclusion = (log(rt) < mean(log(rt)) - 2sd(log(rt))|(log(rt) > mean(log(rt)) + 2sd(log(rt))))) %>% ungroup() %>% filter(exclusion==FALSE)
This results in 238 trials being removed from the 5580 we got after the by-participant exclusions. We now have 5342 trials we can use for analysis.
Now that we have only the rows we want, let’s add some new columns, which will contain important information for each data point. Here, we will be adding:
Ideally, I would’ve added all of these but the first when I actually created the stimuli and logged responses, but I forgot to! Luckily, R allows us to do this post-hoc fairly straightforwardly… which is good, since these features will be critical in our data visualization and analysis.
The question under investigation here is whether or not individuals’ conceptions of gender affect how they process gendered and gender-neutral forms of English personal and professional titles.
In order to examine this, we need to quanify participants’ ideological views! Here we have adopted the 13-item Social Roles Questionnaire put forth in Baber & Tucker (2006). Questions 1-5 correlate to the ‘Gender Transcendent’ subscale, and questions 6-13 correspond to the ‘Gender Linked’ subscale. Each item is scored on a scale of 0-100. So, the first thing we want to do is make two lists of columns which correspond to these two subscales, since the questions are stored individually in the data:
gender_transcendence_cols <- c('subject_information.gender_q1','subject_information.gender_q2','subject_information.gender_q3','subject_information.gender_q4','subject_information.gender_q5')
gender_linked_cols <- c('subject_information.gender_q6','subject_information.gender_q7','subject_information.gender_q8','subject_information.gender_q9','subject_information.gender_q10','subject_information.gender_q11','subject_information.gender_q12','subject_information.gender_q13')
Now we can use the mutate() method on all_data to add two new columns, one for each subscale. We tell R to take the means of the specified columns in [column_names] of all_data for each individual row: rowMeans(all_data[column_names]). We also have to subtract this mean from 100 in the case of the ‘Gender Transcendent’ subscale, since it is inversely scored. Finally, we can create an average total score regardless of subscores, simply by meaning the two subscores we already have.
all_data <- all_data %>%
mutate(gender_trans = 100 - (rowMeans(all_data[gender_transcendence_cols]))) %>%
mutate(gender_link = rowMeans(all_data[gender_linked_cols]))
gender_all = c('gender_trans','gender_link')
all_data <- all_data %>%
mutate(gender_total = rowMeans(all_data[gender_all]))
We also want to add whether the trial included a female or male referent (but also, like, destroy the binary!). In order to do this, we’ll just add a trial_gender column that says ‘female’ if the condition was either ‘neutral_female’ or ‘congruent_female’. Otherwise, we want the trial_gender to say ‘male’.
all_data <- all_data %>%
mutate(trial_gender = ifelse(condition=='neutral_female' | condition == 'congruent_female','female','male'))
all_data %>%
select(workerid,rt,condition,trial_id,trial_gender)
Now we want to add whether or not the lexeme’s neutral form is developed by compounding (as in ‘congress-person’) or by the adoption of the male form (as in ‘actor’ being used more for both men and women). In this study, we only have six lexemes of the latter type, so we’ll just tell R to assign those a morph_type value of ‘adoption’ (for ‘male adoption’), and all else will be assigned a value of ‘compound’.
all_data <- all_data%>%
mutate(morph_type = ifelse(lexeme!= 'actor' & lexeme!= 'host' & lexeme !='hunter' & lexeme!= 'villain' & lexeme!= 'heir' & lexeme!= 'hero','compound','adoption'))
all_data %>%
select(rt,lexeme,morph_type)
Another important factor we want to explore is the length of the critical item! In order to add this, we simply create a new column form_length and tell R to input as that column’s value the length of the string that appears in that row’s form column, which corresponds to the orthograpic form of the critical item in that trial. Note that this will include spaces in the count!
all_data <- all_data %>%
mutate(form_length = str_length(form))
simple_model <- lm(log(rt)~form_length, data = all_data)
all_data <- all_data %>%
mutate(resid_rt = resid(simple_model))
summary(simple_model)
Call:
lm(formula = log(rt) ~ form_length, data = all_data)
Residuals:
Min 1Q Median 3Q Max
-0.8837 -0.3067 -0.0704 0.2480 3.1990
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.969210 0.023635 294.863 < 2e-16 ***
form_length 0.011369 0.002428 4.683 2.95e-06 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.4301 on 2980 degrees of freedom
Multiple R-squared: 0.007305, Adjusted R-squared: 0.006972
F-statistic: 21.93 on 1 and 2980 DF, p-value: 2.954e-06
Now that we have these, we can run a simple linear regression which will show us the effect of orthographic length on reading time. Then we add a new column in the data which is the residual reading time, or the reading time in log space AFTER we control for the effects of orthographic length.
We also want to make sure we have a column which records whether or not the trial was gender-congruent (as in ‘Shelby is a congresswoman’) or gender neutral (as in ‘Shelby is a congressperson’). We add a trial_congruency column, which is valued as ‘congruent’ if that row’s condition is one of the two congruent conditions. Otherwise, it gets valued as ‘neutral’.
all_data <- all_data %>%
mutate(trial_congruency = ifelse(condition=='congruent_male' | condition == 'congruent_female','congruent','neutral'))
Finally, we can classify participants by their particular political alignment; we can construe this broadly as “Republicans” vs. “Democrats”, with those who declined to state a preference, or placed themselves in the middle, as “Non-Partisan”.
all_data <- all_data %>%
mutate(poli_party = ifelse(subject_information.party_alignment == 1 | subject_information.party_alignment == 2,'Republican',ifelse(subject_information.party_alignment == 4 | subject_information.party_alignment == 5,'Democrat','Non-Partisan')))
Now we can start analysing the data by means of data visualization.
inauguration_2021 = c("#5445b1", "#749dae", "#f3c483", "#5c1a33", "#cd3341","#f7dc6a")
ggplot(all_data, aes(x=subject_information.age, y=gender_total)) +
geom_point(alpha=.5) +
geom_smooth(method = 'lm', size=1.2)
ggplot(all_data, aes(x=gender_total, y=resid_rt, color=trial_congruency)) +
geom_point(alpha=.5) +
geom_smooth(method = 'lm', size=1.2) +
theme_minimal()
agg_speaker_mean_con <- all_data %>%
group_by(condition,workerid) %>%
summarize(MeanRT=mean(resid_rt))
`summarise()` has grouped output by 'condition'. You can override using the `.groups` argument.
all_data %>%
group_by(condition,trial_gender) %>%
summarize(MeanRT = mean(resid_rt), CI.Low = ci.low(resid_rt), CI.High = ci.high(resid_rt)) %>%
mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High) %>%
ggplot(aes(x=condition,y=MeanRT,color=trial_gender)) +
geom_point(size=3) +
geom_jitter(data = agg_speaker_mean_con, aes(y=MeanRT),alpha=.1,color='darkred') +
geom_errorbar(aes(ymin=YMin,ymax=YMax), width=.25) +
scale_color_manual(values = inauguration_2021) +
theme_minimal()
`summarise()` has grouped output by 'condition'. You can override using the `.groups` argument.
all_data %>%
group_by(condition,trial_gender,trial_congruency,lexeme) %>%
summarize(MeanRT = mean(resid_rt), CI.Low = ci.low(resid_rt), CI.High = ci.high(resid_rt)) %>%
mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High) %>%
ggplot(aes(x=condition,y=MeanRT,color=trial_gender,shape=trial_congruency)) +
geom_point(size=3) +
geom_errorbar(aes(ymin=YMin,ymax=YMax), width=.25) +
facet_wrap(~ lexeme) +
theme(axis.text.x = element_text(angle = 45, vjust = .7, hjust=.7)) +
scale_color_manual(values = inauguration_2021) +
facet_wrap(~lexeme)
`summarise()` has grouped output by 'condition', 'trial_gender', 'trial_congruency'. You can override using the `.groups` argument.
all_data %>%
filter(morph_type == "adoption") %>%
group_by(condition,trial_gender,trial_congruency,lexeme) %>%
summarize(MeanRT = mean(resid_rt), CI.Low = ci.low(resid_rt), CI.High = ci.high(resid_rt)) %>%
mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High) %>%
ggplot(aes(x=condition,y=MeanRT,color=trial_gender,shape=trial_congruency)) +
geom_point(size=3) +
geom_errorbar(aes(ymin=YMin,ymax=YMax), width=.25) +
facet_wrap(~ lexeme) +
theme(axis.text.x = element_text(angle = 45, vjust = .7, hjust=.7)) +
scale_color_manual(values = inauguration_2021) +
facet_wrap(~lexeme)
`summarise()` has grouped output by 'condition', 'trial_gender', 'trial_congruency'. You can override using the `.groups` argument.
all_data %>%
filter(morph_type == "compound") %>%
group_by(condition,trial_gender,trial_congruency,lexeme) %>%
summarize(MeanRT = mean(resid_rt), CI.Low = ci.low(resid_rt), CI.High = ci.high(resid_rt)) %>%
mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High) %>%
ggplot(aes(x=condition,y=MeanRT,color=trial_gender,shape=trial_congruency)) +
geom_point(size=3) +
geom_errorbar(aes(ymin=YMin,ymax=YMax), width=.25) +
facet_wrap(~ lexeme) +
theme(axis.text.x = element_text(angle = 45, vjust = .7, hjust=.7)) +
scale_color_manual(values = inauguration_2021) +
facet_wrap(~lexeme)
`summarise()` has grouped output by 'condition', 'trial_gender', 'trial_congruency'. You can override using the `.groups` argument.
temp <- all_data %>%
group_by(trial_gender) %>%
summarize(MeanRT = mean(rt), CI.Low = ci.low(rt), CI.High = ci.high(rt)) %>%
mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High)
dodge = position_dodge(.9)
ggplot(data=temp, aes(x=trial_gender,y=MeanRT,fill=trial_gender)) +
geom_bar(stat='identity',position=dodge) +
geom_errorbar(aes(ymin=YMin,ymax=YMax),width=.25,position=dodge) +
theme(legend.position = 'none')
dodge = position_dodge(.9)
all_data %>%
group_by(trial_gender) %>%
summarize(MeanRT = mean(resid_rt)) %>%
ggplot(aes(x=trial_gender,y=MeanRT,fill=trial_gender)) +
geom_bar(stat='identity',position=dodge) +
theme(legend.position = 'none')
dodge = position_dodge(.9)
all_data %>%
group_by(trial_congruency) %>%
summarize(MeanRT = mean(resid_rt)) %>%
ggplot(aes(x=trial_congruency,y=MeanRT,fill=trial_congruency)) +
geom_bar(stat='identity',position=dodge) +
theme(legend.position = 'none')
all_data %>%
group_by(condition,trial_gender,morph_type) %>%
summarize(MeanRT = mean(resid_rt), CI.Low = ci.low(resid_rt), CI.High = ci.high(resid_rt)) %>%
mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High) %>%
ggplot(aes(x=condition,y=MeanRT,color=trial_gender)) +
geom_point(size=3) +
geom_errorbar(aes(ymin=YMin,ymax=YMax), width=.25) +
scale_color_manual(values = inauguration_2021) +
facet_wrap(~morph_type) +
theme(axis.text.x = element_text(angle=45, vjust = 0.5))
`summarise()` has grouped output by 'condition', 'trial_gender'. You can override using the `.groups` argument.
agg_speaker_trial <- all_data %>%
group_by(condition,workerid) %>%
summarize(MeanRT=mean(resid_rt))
`summarise()` has grouped output by 'condition'. You can override using the `.groups` argument.
test_model <- lm(resid_rt~trial_congruency*morph_type, data=all_data)
summary(test_model)
Call:
lm(formula = resid_rt ~ trial_congruency * morph_type, data = all_data)
Residuals:
Min 1Q Median 3Q Max
-0.9005 -0.3021 -0.0725 0.2436 3.1904
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.04198 0.02044 2.054 0.040111 *
trial_congruencyneutral -0.03333 0.02864 -1.164 0.244666
morph_typecompound -0.10134 0.02434 -4.164 3.22e-05 ***
trial_congruencyneutral:morph_typecompound 0.13118 0.03422 3.834 0.000129 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.4279 on 2978 degrees of freedom
Multiple R-squared: 0.01097, Adjusted R-squared: 0.009969
F-statistic: 11.01 on 3 and 2978 DF, p-value: 3.478e-07
test_model2 <- lm(resid_rt~trial_gender, data=all_data)
summary(test_model2)
Call:
lm(formula = resid_rt ~ trial_gender, data = all_data)
Residuals:
Min 1Q Median 3Q Max
-0.8799 -0.3054 -0.0716 0.2498 3.1953
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.003744 0.011138 -0.336 0.737
trial_gendermale 0.007488 0.015751 0.475 0.635
Residual standard error: 0.4301 on 2980 degrees of freedom
Multiple R-squared: 7.583e-05, Adjusted R-squared: -0.0002597
F-statistic: 0.226 on 1 and 2980 DF, p-value: 0.6345
compounds_only <- all_data %>%
filter(morph_type == 'compound')
compounds_only <- compounds_only %>%
mutate(ctrial_congruency = as.numeric(as.factor(trial_congruency))-mean(as.numeric(as.factor(trial_congruency)))) %>%
mutate(ctrial_gender = as.numeric(as.factor(trial_gender))-mean(as.numeric(as.factor(trial_gender)))) %>%
mutate(cgender_link = scale(gender_link)) %>%
mutate(cgender_total = scale(gender_total))
compound_model <- lmer(resid_rt~ctrial_congruency*ctrial_gender*cgender_total + (1|workerid) + (1|lexeme) + (1|name),data = compounds_only)
boundary (singular) fit: see ?isSingular
summary(compound_model)
Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: resid_rt ~ ctrial_congruency * ctrial_gender * cgender_total +
(1 | workerid) + (1 | lexeme) + (1 | name)
Data: compounds_only
REML criterion at convergence: 1516.9
Scaled residuals:
Min 1Q Median 3Q Max
-2.9463 -0.6196 -0.1141 0.5120 5.1743
Random effects:
Groups Name Variance Std.Dev.
workerid (Intercept) 0.072802 0.26982
name (Intercept) 0.000000 0.00000
lexeme (Intercept) 0.009709 0.09853
Residual 0.097840 0.31279
Number of obs: 2089, groups: workerid, 152; name, 24; lexeme, 14
Fixed effects:
Estimate Std. Error df t value Pr(>|t|)
(Intercept) -0.01169 0.03492 33.77863 -0.335 0.7398
ctrial_congruency 0.10120 0.01386 1927.09399 7.301 4.16e-13 ***
ctrial_gender 0.02769 0.01387 1927.54232 1.996 0.0461 *
cgender_total 0.05201 0.02294 150.06294 2.267 0.0248 *
ctrial_congruency:ctrial_gender -0.01774 0.02780 1929.12259 -0.638 0.5234
ctrial_congruency:cgender_total 0.01692 0.01386 1926.12747 1.221 0.2224
ctrial_gender:cgender_total -0.02024 0.01389 1927.94098 -1.457 0.1452
ctrial_congruency:ctrial_gender:cgender_total -0.04153 0.02776 1927.38754 -1.496 0.1349
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr) ctrl_c ctrl_g cgndr_ ctrl_cngrncy:ct_ ctrl_cngrncy:cg_ ctrl_g:_
ctrl_cngrnc 0.000
ctrial_gndr 0.000 -0.008
cgender_ttl 0.000 0.003 -0.001
ctrl_cngrncy:ct_ -0.002 0.001 0.003 0.001
ctrl_cngrncy:cg_ 0.002 -0.006 0.003 0.009 -0.002
ctrl_gndr:_ -0.001 0.005 0.004 0.000 0.016 -0.003
ctrl_cn:_:_ 0.001 0.001 0.016 -0.001 0.000 0.000 0.034
optimizer (nloptwrap) convergence code: 0 (OK)
boundary (singular) fit: see ?isSingular
adoptions_only <- all_data %>%
filter(morph_type == 'adoption')
adoptions_only <- adoptions_only %>%
mutate(ctrial_congruency = as.numeric(as.factor(trial_congruency))-mean(as.numeric(as.factor(trial_congruency)))) %>%
mutate(ctrial_gender = as.numeric(as.factor(trial_gender))-mean(as.numeric(as.factor(trial_gender)))) %>%
mutate(cgender_link = scale(gender_link)) %>%
mutate(cgender_total = scale(gender_total))
adoptions_model <- lmer(resid_rt~ctrial_congruency*ctrial_gender*cgender_total + (1|workerid) + (1|lexeme) + (1|name),data = adoptions_only)
boundary (singular) fit: see ?isSingular
summary(adoptions_model)
Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: resid_rt ~ ctrial_congruency * ctrial_gender * cgender_total +
(1 | workerid) + (1 | lexeme) + (1 | name)
Data: adoptions_only
REML criterion at convergence: 780.7
Scaled residuals:
Min 1Q Median 3Q Max
-2.2764 -0.6128 -0.1019 0.4977 7.0825
Random effects:
Groups Name Variance Std.Dev.
workerid (Intercept) 0.05999 0.2449
name (Intercept) 0.00000 0.0000
lexeme (Intercept) 0.02594 0.1611
Residual 0.10261 0.3203
Number of obs: 893, groups: workerid, 152; name, 24; lexeme, 6
Fixed effects:
Estimate Std. Error df t value Pr(>|t|)
(Intercept) 0.026404 0.069521 5.924863 0.380 0.71734
ctrial_congruency -0.026661 0.022600 773.237762 -1.180 0.23848
ctrial_gender -0.024247 0.022594 773.262319 -1.073 0.28353
cgender_total 0.061983 0.022598 148.801276 2.743 0.00684 **
ctrial_congruency:ctrial_gender -0.033364 0.045588 779.086714 -0.732 0.46447
ctrial_congruency:cgender_total -0.009892 0.022408 764.311685 -0.441 0.65901
ctrial_gender:cgender_total -0.011764 0.022586 769.611878 -0.521 0.60261
ctrial_congruency:ctrial_gender:cgender_total 0.039454 0.045136 767.084621 0.874 0.38232
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr) ctrl_c ctrl_g cgndr_ ctrl_cngrncy:ct_ ctrl_cngrncy:cg_ ctrl_g:_
ctrl_cngrnc 0.000
ctrial_gndr 0.000 0.029
cgender_ttl -0.001 -0.008 0.002
ctrl_cngrncy:ct_ 0.003 0.001 0.005 -0.008
ctrl_cngrncy:cg_ -0.003 -0.021 -0.013 -0.031 0.020
ctrl_gndr:_ 0.001 -0.013 0.002 0.003 -0.005 0.007
ctrl_cn:_:_ -0.003 0.020 -0.003 0.001 -0.007 0.011 -0.057
optimizer (nloptwrap) convergence code: 0 (OK)
boundary (singular) fit: see ?isSingular
all_data <- all_data %>%
mutate(ctrial_congruency = as.numeric(as.factor(trial_congruency))-mean(as.numeric(as.factor(trial_congruency)))) %>%
mutate(ctrial_gender = as.numeric(as.factor(trial_gender))-mean(as.numeric(as.factor(trial_gender)))) %>%
mutate(cgender_link = scale(gender_link)) %>%
mutate(cgender_total = scale(gender_total)) %>%
mutate(cmorph_type = as.numeric(as.factor(morph_type))-mean(as.numeric(as.factor(morph_type))))
complex_model <- lmer(resid_rt~ctrial_congruency*ctrial_gender*cgender_total + (1|workerid) + (1|lexeme) + (1|name),data = all_data)
boundary (singular) fit: see ?isSingular
summary(complex_model)
Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: resid_rt ~ ctrial_congruency * ctrial_gender * cgender_total +
(1 | workerid) + (1 | lexeme)
Data: all_data
REML criterion at convergence: 2129
Scaled residuals:
Min 1Q Median 3Q Max
-2.5602 -0.6318 -0.1277 0.5143 7.7057
Random effects:
Groups Name Variance Std.Dev.
workerid (Intercept) 0.06833 0.2614
lexeme (Intercept) 0.01362 0.1167
Residual 0.10093 0.3177
Number of obs: 2982, groups: workerid, 152; lexeme, 20
Fixed effects:
Estimate Std. Error df t value Pr(>|t|)
(Intercept) -1.370e-04 3.413e-02 4.764e+01 -0.004 0.9968
ctrial_congruency 5.971e-02 1.166e-02 2.806e+03 5.120 3.27e-07 ***
ctrial_gender 9.664e-03 1.166e-02 2.806e+03 0.829 0.4075
cgender_total 5.441e-02 2.199e-02 1.500e+02 2.474 0.0145 *
ctrial_congruency:ctrial_gender -1.767e-02 2.334e-02 2.807e+03 -0.757 0.4490
ctrial_congruency:cgender_total 1.010e-02 1.167e-02 2.807e+03 0.865 0.3870
ctrial_gender:cgender_total -1.820e-02 1.168e-02 2.807e+03 -1.559 0.1191
ctrial_congruency:ctrial_gender:cgender_total -1.689e-02 2.335e-02 2.807e+03 -0.723 0.4695
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr) ctrl_c ctrl_g cgndr_ ctrl_cngrncy:ct_ ctrl_cngrncy:cg_ ctrl_g:_
ctrl_cngrnc 0.000
ctrial_gndr 0.000 0.000
cgender_ttl -0.001 0.001 -0.001
ctrl_cngrncy:ct_ 0.000 0.000 0.001 0.000
ctrl_cngrncy:cg_ 0.001 -0.001 -0.003 0.001 -0.003
ctrl_gndr:_ 0.000 -0.001 0.001 0.000 0.002 -0.001
ctrl_cn:_:_ 0.000 -0.001 0.003 0.000 0.001 0.000 0.004
ideology_model <- lmer(resid_rt~ctrial_congruency*ctrial_gender*cgender_link + (1|workerid) + (1|lexeme) + (1|name), data=all_data)
boundary (singular) fit: see ?isSingular
summary(ideology_model)
Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: resid_rt ~ ctrial_congruency * ctrial_gender * cgender_link +
(1 | workerid) + (1 | lexeme) + (1 | name)
Data: all_data
REML criterion at convergence: 2129.1
Scaled residuals:
Min 1Q Median 3Q Max
-2.5813 -0.6321 -0.1304 0.5107 7.6659
Random effects:
Groups Name Variance Std.Dev.
workerid (Intercept) 0.06754 0.2599
name (Intercept) 0.00000 0.0000
lexeme (Intercept) 0.01365 0.1168
Residual 0.10099 0.3178
Number of obs: 2982, groups: workerid, 152; name, 24; lexeme, 20
Fixed effects:
Estimate Std. Error df t value Pr(>|t|)
(Intercept) -1.209e-04 3.407e-02 4.720e+01 -0.004 0.99718
ctrial_congruency 5.968e-02 1.167e-02 2.806e+03 5.116 3.33e-07 ***
ctrial_gender 9.695e-03 1.167e-02 2.806e+03 0.831 0.40607
cgender_link 6.112e-02 2.184e-02 1.500e+02 2.798 0.00581 **
ctrial_congruency:ctrial_gender -1.761e-02 2.334e-02 2.807e+03 -0.754 0.45070
ctrial_congruency:cgender_link 1.229e-02 1.167e-02 2.807e+03 1.053 0.29254
ctrial_gender:cgender_link -1.021e-02 1.168e-02 2.807e+03 -0.874 0.38218
ctrial_congruency:ctrial_gender:cgender_link -6.519e-03 2.336e-02 2.807e+03 -0.279 0.78017
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr) ctrl_c ctrl_g cgndr_ ctrl_cngrncy:ct_ ctrl_cngrncy:cg_ ctrl_g:_
ctrl_cngrnc 0.000
ctrial_gndr 0.000 0.000
cgender_lnk 0.000 0.001 -0.001
ctrl_cngrncy:ct_ 0.000 0.000 0.001 0.000
ctrl_cngrncy:cg_ 0.001 -0.001 -0.002 0.001 -0.002
ctrl_gndr:_ 0.000 -0.001 0.000 0.000 0.002 -0.001
ctrl_cn:_:_ 0.000 0.000 0.003 0.000 0.000 0.000 0.004
optimizer (nloptwrap) convergence code: 0 (OK)
boundary (singular) fit: see ?isSingular
new_toy <- read.csv('maze_task_1-merged.csv') %>%
filter(trial_id!= 'example') %>%
filter(response_correct != 0) %>%
mutate(trial_gender = ifelse(condition=='neutral_female' | condition == 'congruent_female','female','male')) %>%
mutate(trial_congruency = ifelse(condition=='congruent_male' | condition == 'congruent_female','congruent','neutral'))
new_toy %>%
group_by(word_idx,trial_gender) %>%
summarize(MeanRT = mean(rt)) %>%
ggplot(aes(x=word_idx, y=log(MeanRT), color=trial_gender)) +
geom_line() +
geom_point()
`summarise()` has grouped output by 'word_idx'. You can override using the `.groups` argument.
new_toy %>%
group_by(word_idx,trial_gender) %>%
summarize(MeanRT = mean(rt)) %>%
ggplot(aes(x=word_idx, y=MeanRT, color=trial_gender)) +
geom_line() +
geom_point()
`summarise()` has grouped output by 'word_idx'. You can override using the `.groups` argument.
Rscript merge_results.R maze_task_1-merged.csv dems maze_task_2-merged.csv reps
Part of this experiment and analysis was also carried out as part of my class project in Stanford’s LINGUIST 245B ‘Methods in Psycholinguistics’ class, taught by Judith Degen.↩︎
This amounts to an hourly rate of $20.73. We originally anticipated that participants would take an average of 7 minutes to complete the experiment, and set the base pay at $15 an hour.↩︎
7 Comments